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ABSTRACT 

Objective: To evaluate the 
satisfaction of the intensive care unit 
staif with a computerized physician 
order entry and to compare the 
concept of the computerized physician 
order entry relevance among intensive 
care unit healthcare workers. 

Methods: We performed a 
cross-sectional survey to assess the 
satisfaction of the intensive care unit 
staif with the computerized physician 
order entry in a 30-bed medical/ surgical 
adult intensive care unit using a 
self-administered questionnaire. The 
questions used for grading satisfaction 
levels were answered according to a 
numerical scale that ranged from 1 
point (low satisfaction) to 10 points 
(high satisfaction). 

Results: The majority of the 
respondents (n=250) were female 
(66%) between the ages of 30 and 
35 years of age (69%). The overall 
satisfaction with the computerized 
physician order entry scored 5.74±2.14 
points. The satisfaction was lower 
among physicians (n=42) than among 
nurses, nurse technicians, respiratory 
therapists, clinical pharmacists and diet 
specialists (4.62±1.79 versus 5.97±2.14, 
p<0.001); satisfaction decreased with 



age (p<0.001). Physicians scored 
lower concerning the potential of the 
computerized physician order entry for 
improving patient safety (5.45±2.20 
versus 8.09±2.21, p<0.001) and 
the ease of using the computerized 
physician order entry (3.83±1.88 
versus 6.44±2.31, p<0.001). The 
characteristics independently associated 
with satisfaction were the system's 
user-friendliness, accuracy, capacity 
to provide clear information, and fast 
response time. 

Conclusion: Six months after its 
implementation, healthcare workers 
were satisfied, albeit not entirely, with 
the computerized physician order entry. 
The overall users' satisfaction with 
computerized physician order entry was 
lower among physicians compared to 
other healthcare professionals. The factors 
associated with satisfaction included the 
belief that digitalization decreased the 
workload and contributed to the intensive 
care unit quality with a user-friendly and 
accurate system and that digitalization 
provided concise information within a 
reasonable time frame. 
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INTRODUCTION 

In the intensive care unit (ICU), medication errors 
commonly occur because of the severity of illnesses, the 
complexity of care, and the frequency of daily changes in 
medication orders.''^' Medication errors are associated 
with increases in mortality and morbidity, length of 
hospital stay, costs, stress of the healthcare professional 
implicated in this situation, and can potentially ruin an 
institution's reputation.*^ 

Errors can occur at any stage of the medication 
process, from ordering the medications to administering 
the medications; therefore, any healthcare professional 
involved with patients' medications can be liable. The 
computerized physician order entry (CPOE) reduces, by 
more than 50%, the incidence of medication errors,'^'** 
especially when the CPOE uses bar-code technology to 
verify that the correct medication is administered to the 
correct patient.'" The CPOE also allows information to 
be presented in a structured format (dose, administration 
route, and interval) and to be readable and electronically 
accessible to all of the staff members. Additionally, 
healthcare professionals can easily check the order for 
drug interactions, dose errors, side effects and allergic 
reactions.""' 

Despite these potential benefits, changing from 
the paper medication orders to the CPOE is complex 
and can be perturbing because the transition requires 
effort and engagement among all of the professionals 
who are involved in the medication process. Resistance, 
dissatisfaction, conflict, and stress can exist among the 
healthcare professionals during the adaptation phase. 
To implement the CPOE, hospitals must commit large 
financial investments for acquiring or developing the 
required technology. 

Few studies in the literature evaluate user satisfaction 
with the CPOE, particularly concerning all users' 
evaluation in an ICU environment.*'^""* The primary 
objectives of this study were to evaluate the satisfaction 
of the ICU physician staff and other healthcare workers 
(nurses, nurse technicians, respiratory therapists, clinical 
pharmacists and diet specialists) with a CPOE and to 
compare the concept of the CPOE's relevance among 
the ICU healthcare workers after six months of CPOE 
implementation in an ICU. 

METHODS 

The present study was conducted in a 30-bed, adult 
medical-surgical ICU in a tertiary private hospital in Sao 



Paulo, Brazil, six months after implementing a CPOE 
(Tasy -Wheb Sistemas/Philips, Blumenau, SC, Brazil) 
in January 20 1 1 . The around-the-clock professional/bed 
ratios in the ICU are as follows: nurse 1 :4; nurse-technician 
1:2; physician 1:6 (work hours) and 1:10 (night shifts). 

All of the ICU professionals who used the CPOE and 
took care of patients were invited to participate in the 
study (physicians, nurses, nurse technicians, respiratory 
therapists, clinical pharmacists, and diet specialists). They 
were excluded from the study if they worked for less than 
six months in the ICU. Participation was voluntary, and 
the self-administered questionnaire was anonymously 
returned in a sealed envelope. Informed consent to 
participate in the study was given by all professionals 
using the standardized hospital consent form, including 
the consent to publish. The study was approved by 
the Research and Ethics Committee of the Hospital 
Strio-Libanes (protocol #HSL 201 1/18). 

This CPOE comprises various stages of the 
medication process: medical order; pharmaceutical 
evaluation; medication preparation and dispensing; 
administration of the medications by a nurse or a 
nurse technician; and use of a bar code system by the 
pharmacist, nurse and nurse technician. The details of 
the medication process are described in the electronic 
supplementary materials (ESM). 

We used an adapted satisfaction questionnaire"^' 
to evaluate the satisfaction of physicians and other 
healthcare workers using the CPOE system. The 
questions for grading the satisfaction levels were 
answered in a numerical scale that ranged from 1 point 
(low satisfaction) to 10 points (high satisfaction). The 
questionnaire was composed of 10 general questions that 
applied to all professionals, 10 specific questions applied 
only to physicians, and 4 specific questions applied 
only to nurses and nurse technicians (available in the 
ESM). Age was categorized as <30, 30-35, 36-50 and 
>50 years old. The healthcare professionals were asked 
about the frequency of computer use outside of the work 
environment: daily, weekly, monthly, or not at all. 

Statistical analysis 

Numerical variables were expressed as the mean and 
standard deviation, and categorical variables were expressed 
as frequencies and percentages. The differences in the 
satisfaction scores between groups were tested by AN OVA 
followed by a Bonferroni post hoc test. A ^-test was used 
to compare physicians with other healthcare professionals. 
A p<0.05 was considered to be statistically significant. To 
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evaluate which aspects determined the CPOE satisfaction, 
univariate Unear regressions were performed using age 
category, gender, profession, computer use outside of 
work hours, and each question. The questions associated 
with the CPOE satisfaction (p<0. 10) were incorporated 
into the multivariate linear regression model. We used a 
backward stepwise selection to identify which questions 
were independently associated with CPOE satisfaction. 
The analyses were performed using Statistical Package for 
the Social Sciences (SPSS) software version 11.1 and R 
version 2.9.2 (R Development Core Team). 

RESULTS 

Between July 2011 and November 2011, all of the 
ICU healthcare workers were invited to participate in the 
study, and all of them answered the questionnaire. The 
majority of the respondents were female (66%), ranging 
in age between 30 years and 35 years (69%), and used 
computers every day at home (81%). On average, the 
satisfaction level decreased with age (effect size=-0.55; 
p<0.001). 

Figure 1 shows the mean scores from the different 
groups regarding the global satisfaction with the CPOE. 
Notably, when all of the respondents were computed 
together, the overall satisfaction with the CPOE scored 
5.74±2.14 points. The physicians gave the highest score 
to the ease of duplicating the medical order (6.39±1.93) 
and the lowest score to the ease of accessing and copying 
the patients' previous medical histories (3.85±2.02) 
(Figure 2). 
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Figure 1 - Global satisfation with the computerized physician order entry. The 
figure presents the scores for each individual group and for all intensive care units 
professionals computed together. ICU - intensive care unit. l=low satisfaction 
and 10=high satisfaction. 



The nurses and nurse technicians gave automatic 
medication scheduling a score of 5.99±2.58 and gave a 
score of 5.95±2.5 1 to the electronic checking of medication 
administration using the bar code scanner. The system's 
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Figure 2 - Visualization of the scores given by physicians to specific factors 
related to the satisfaction with the computerized physician order entry system. 
1 =low satisfaction and 10 = high satisfaction. 

ability to provide clear and correct information scored 
5.86±2.46, and the reception in the unit of the items 
dispensed by the pharmacy scored 5.37±2.32. 

Table 1 shows the comparison of mean scores between 
the physicians and other healthcare professionals. The 
satisfaction score was lower among physicians, who found 
the system to be less user-friendly. Table 1 in the ESM 
shows the differences in the satisfaction scores between 
groups. The ICU staff gave the highest score to the increase 
in patients' safety after the CPOE implementation 
(7.64±2.42 points). 

Table 2 shows the factors that were significantly 
associated with satisfaction with the CPOE in the 
univariate analyses. After multivariate adjustment, the 
factors that remained significant were the belief that 
digitalization decreased the workload and contributed to 
ICU quality and a user-friendly and accurate system and 
that digitalization provided concise information within a 
reasonable time frame. 

DISCUSSION 

This study was conducted six months after 
implementing the electronic system in our medical/ 
surgical adult ICU. We noted that health care professionals 
were neither extremely dissatisfied nor entirely satisfied 
with the CPOE according to the values used for 
categorizing satisfaction in the literature: 1 -3 correspond to 
extreme dissatisfaction, 4-6 correspond to dissatisfaction, 
7-8 correspond to satisfaction, and 9-10 correspond to 
entire satisfaction."^* The respondents identified the most 
relevant CPOE characteristics as follows: user-friendliness, 
accuracy, clarity of information, fast response time; all 
of these characteristics contributed to decreasing the 
workload and improving the ICU quality. 
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Table 1 - Comparison of mean scores between the physicians and all other professionals for the assessment of the satisfaction with the computerized physician order 
entry system 





Physicians (N=42) Mean (SD) 


All otber professionals (N=208) Mean (SD) 


Mest 


Satisfaction with the digitalization of the information in the hospital 


5.88±1.85 




7.14±1.81 


<0.001 


Satisfaction with the digitalization of the information in the ICU 


5.71 ±1.93 




7.01 ±1.95 


<0.001 


Satisfaction with the computerized physician enter ordering system 


4.B2±1.79 




5.97±2.14 


<0.001 


Does the CPOE make the daily workflow easier or more difficult? 


5.98±2.45 




7.31 ±2.37 


0.001 


The system provides security for the patient 


5.45±2.20 




8.09±2.21 


<0.001 


The system contributes to the quality of care in the ICU 


5.40±2.18 




7.53±2.31 


<U.UU 1 


The system is user-friendly 


3.88±1.85 




6.40±2.29 


<0.001 


The system is accurate 


5.17±1.77 




B.77±2.34 


<0.001 


The system provides clear information 


4.76±1.72 




B.3B±2.28 


<0.001 


Do you get the information you need in time? 


4.69±1.96 




5.97±2.36 


0.001 


The system is easy to use 


3.83±1.89 




B.44±2.31 


<0.001 


SD - standard deviation; ICU - intensive care unit; CPOE - computerized physician order entry. 1 =low satisfaction to 10=high satisfaction. 




Table 2 - Univariate and multivariate analysis of factors associated with satisfaction with the computerized physician order entry system 






Univariate 




Multivariate 




Questions applied to all ICU professionals 


Regression coefficient 


p value 


Regression coefficient 


p value 


Satisfaction with the digitalization of the information in the hospital 


0.57 


<0.0001 




- 


The digitalization of the information facilitates your work 


0.57 


<0.0001 


0.13 


0.004 


The system contributes to patient safety 


0.51 


<0.0001 






The system contributes to the ICU quality 


0.B4 


<0.0001 


0.17 


O.OOB 


The system is user-friendly 


0.70 


<0.0001 


0.24 


<0.001 


The system is accurate 


0.B9 


<0.0001 


0.12 


0.040 


The system provides clear information 


0.72 


<0.0001 


0.20 


0.001 


The system provides information in time 


0.B8 


<0.0001 


0.15 


0.003 


The system is easy to use 


0.B4 


<0.0001 







ICU - intensive care unit. 



Although there are many benefits for using of CPOE 
in an ICU, difficulties may occur during and after 
CPOE implementation. Studies conducted in different 
countries regarding the use of diverse CPOE systems 
have demonstrated that the number of medication errors 
can increase in the initial months after implementing 
the system during the adaptation period.*"* Health 
care professionals, especially physicians, are resistant to 
changing their routines; the majority of professionals 
continue to be adamant against using electronic systems 
in the hospital, which constitutes one of the barriers 
to CPOE adoption.*^"* Physicians are likely to be not 
as satisfied as non-physicians because they fear that the 
change from paper to CPOE might affect patient care, 
particularly because the medical order constitutes the core 
of the medication process. 



We found an inverse relationship between satisfaction 
and age. This finding is in agreement with the findings 
of others, who have shown that older users have more 
difficulties adapting to new technologies and are thus 
more resistant to changing routines.*'^' Moreover, using 
the computer outside of work hours tends to be less 
common with age (p=0. 107, data not shown), which may 
affect the capacity to adapt to the CPOE, thus having an 
impact on satisfaction. 

Our study is the first to assess the impact on end-user 
satisfaction of the combined implementation of CPOE 
and bar-coding with comparisons between different user 
groups in an ICU. CPOE is a technology with the most 
potential for improving medication safety in ICUs'''* '" '" 
because the majority of serious errors in ICUs are related to 
medication.*^'' With CPOE integrated with the bar-code 
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reader at the bedside, it is possible to ensure that the 
correct medication is administered to the correct patient, 
at the correct dose and at the correct time. Furthermore, 
the CPOE systems can alert physicians to patient allergies, 
medication dosing and duplicity. Medication errors can 
be potentially life-threatening in critically ill patients who 
are more vulnerable and more susceptible to those errors 
because their commonly decreased level of consciousness 
impairs their capacity to oversee the treatment delivered to 
them. Additionally, several medications are administered 
out of schedule, and many medications are intravenously 
administered with the need for frequent changes in dose or 
in the rate of administration. The immediate benefit of 
CPOE is the accuracy of the medication order, improving 
patient safety in addition to saving nurses and pharmacists 
from deciphering a doctor's poor handwriting.""'^*' 

We showed a lower satisfaction level with the 
CPOE among physicians compared to other health care 
professionals. This finding is in contrast to other studies 
in which physicians were more satisfied with the CPOE 
than nurses. In our opinion, this inconsistency can 
be explained by the various levels of interaction with the 
system depending on the user category. In our hospital, 
the majority of the workload related to the CPOE 
lies on the physicians who are responsible for the daily 
notes, prescription orders, and ordering of laboratory 
and imaging tests. Similarly, it is not surprising that 
nurse technicians were the most satisfied with the CPOE 
because the bar-code system makes the workflow easier, 
generates more patient safety, and reduces the likelihood 
of errors with minimal additional work. 



There are limitations in our study. First, the study 
was conducted in a single center, and the results may 
reflect characteristics of this particular ICU. Second, 
the questionnaire was not previously validated for this 
specific use. Third, our survey was performed in the 
first six months after the CPOE implementation, and 
some of the dissatisfaction might be related to the initial 
adaptation period. Finally, the time each healthcare 
professional spent using the system for his or her daily 
tasks was not quantified; therefore, we could not consider 
our satisfaction analyses. 

CONCLUSIONS 

In summary, six months after its implementation, 
health care workers were satisfied, albeit not entirely, 
with the computerized physician order entry. The 
characteristics independently associated with satisfaction 
were the system's user-friendliness, accuracy, capability of 
providing clear information, and fast response time. Users 
want a system that can decrease the workload and improve 
the intensive care unit quality. 
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RESUMO 

Objetivo: Avaliar a satisfa^ao da equipe da unidade de terapia 
intensiva com o prontuario eletronico do paciente e comparar a 
relevancia do conceito de registro eletronico de ordens medicas 
entre os profissionais de saiide da unidade de terapia intensiva. 

Metodos: Estudo transversal de levantamento para avaliar 
a satisfa9ao da equipe da unidade de terapia intensiva com o 
prontuario eletronico do paciente em uma unidade de terapia 
intensiva clinica e ciriirgica para pacientes adultos com 30 leitos, 
utilizando um questionario de autopreenchimento. As questoes 
utilizadas para graduar os niveis de satisfa^ao foram respondidas 
segundo uma escala numerica, que variava de 1 (baixo grau de 
satisfa9ao) a 10 pontos (elevado grau de satisfa^ao). 

Resultados: As pessoas que responderam ao questionario 
{n=250) eram, em sua maioria, do genero feminino (66%) com 



idades entre 30 e 35 anos (69%). O grau geral de satisfa^ao com 
o prontuario eletronico do paciente foi de 5,74±2,14 pontos. 
O grau de satisfa^ao foi mais baixo entre os medicos (n=42) 
do que entre enfermeiros, tecnicos de enfermagem, terapeutas 
respiratorios, farmaceuticos clinicos e nutricionistas (4,62±1,79 
em comparafao com 5,97±2,14; p<0,001); o grau de satisfa^ao 
decresceu com a idade (p<0,001). Os medicos tiveram niveis 
mais baixos de satisfa^ao com rela^ao ao potencial do sistema de 
registro eletronico de ordens medicas de melhorar a seguran^a do 
paciente (5,45±2,20 em compara^ao com 8,09±2,21; p<0,001) 
e facilidade de uso do sistema de registro eletronico de ordens 
medicas (3,83±1,88 em compara^ao com 6,44±2,31; p<0,001). 
As caracteristicas com associa^ao independente com satisfa^ao 
foram sistema amigavel ao usuario, precisao, capacidade de 
fornecer informa^ao clara, e tempo de resposta rapido. 
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Conclusao: Depois de 6 meses da implanta^ao do sistema, 
OS profissionais de saiide estavam satisfeitos, embora nao 
totalmente, com o prontuario eletronico do paciente. O grau 
geral de satisfa^ao entre os usuarios do prontuario eletronico 
do paciente foi mais baixo entre os medicos do que entre os 
demais profissionais de saiide. Os fatores associados com o grau 
de satisfa^ao incluiram a cren^a de que a digitaliza^ao reduziria 



a carga de trabalho e contribuiria para qualidade da unidade 
de terapia intensiva com um sistema amigavel e preciso para 
o usuario e que a digitaliza^ao proporcionaria infijrmafoes 
concisas dentro de um periodo de tempo razoavel. 

Descritores: Prontuario eletronico do paciente; Conduta na 
pratica dos medicos; Pesquisas sobre servi^os de saiide; Atitude 
do pessoal de saiide; Satisfa^ao no trabalho 
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